Method and apparatus for sorting

ABSTRACT

A method of sorting is described and which includes a step of acquiring a multiplicity of synchronized image signals of a product stream which is to be sorted; generating a multiplicity of fused sensor signals; forming an image model previously acquired from the objects to be sorted; identifying objects in the product stream, and generating object presence and defect signals; determining a spatial orientation of the objects in the product stream; detecting the defects and removing the defects from the product stream.

RELATED APPLICATIONS

This application is a Continuation Application of co-pending U.S. patentapplication Ser. No. 15/634,694, filed Jun. 27, 2017, entitled “A Methodand Apparatus for Sorting,” for which a Notice of Allowance (NOA) hasbeen issued by the USPTO and which is fully and completely incorporatedherein by reference. This Continuation Patent Application also claimspriority to earlier filed PCT/US2018/014362, having the same title,filed on 19 Jan., 2019 and which is fully and completely incorporatedherein by reference.

The inventorship of this Continuation Patent Application is the same asthe inventorship of co-pending U.S. patent application Ser. No.15/634,694, and of co-pending PCT/US2018/014362 to which thisContinuation Application claims priority.

TECHNICAL FIELD

The present invention relates to a method and apparatus for sorting, andmore specifically to a method for determining a defect in anagricultural product which typically cannot be visually discerned, andthen removing the product having the agricultural defect or the defectitself, from a moving product stream.

BACKGROUND OF THE INVENTION

The developers of optical sorting systems which are uniquely adapted forvisually inspecting a mass-flow of a given food product have endeavored,through the years, to provide increasing levels of information which areuseful in making well-informed sorting decisions to effect sortingoperations in mass-flow food sorting devices. While the capturing andprocessing of product images employing prior art cameras and otheroptical devices has long been known, it has also been recognized thatimages of a product formed by visible spectrum electromagnetic radiationtypically will not provide enough information for an automated sortingmachine to accurately identify all (and especially hidden) food or otheragricultural defects, and which may subsequently be later identifiedafter further processing of the product. For example, one of the defectsin agricultural products which have troubled food processors through theyears has been the effective identification of “sugar end” defects inpotato products, and more specifically potato products that are destinedfor processing into food items such as French fries and the like.

“Sugar ends” and which are also referred to as “dark ends”, “glassyends”, “translucent ends” and “jelly ends” is a physiological, tuberdisorder, which is caused by environmental conditions which occur duringthe growth of the potato plant. Potato strips or fries made from “sugarend” potatoes exhibit or display undesirable dark-brown areas on theproduct after it has been subjected to frying. This defect is typicallycaused by the higher concentration of reducing sugars found in the givendarkened region of the potato. The process of frying the product resultsin caramelizing, which creates the undesirable dark brown region on thefried product. Heretofore, the challenge with food processors has beenthat the “sugar end” defects are typically invisible to traditionaloptical detection technology until after the potato product has beenthoroughly fried. In view of this situation, potato strip processors canbe unaware they have “sugar end” problems with a given lot of potatoesthey are processing until their downstream food service customers frythe potato strips and then provide complaints. “Sugar ends” are usuallyassociated with tubers that have a harvested shape which are somewhatpointed on the stem end of the potato. However, tubers having what isconsidered to be an ideal shape may also develop this anomaly.

Those skilled in the art have recognized that cultural, or managementpractices that increase a potato plant's susceptibility to heat ormoisture stress during tuber initiation, and bulking, can encourage“sugar end” development. As should be understood, tubers areparticularly sensitive to environmental stress during the early bulkingphase. It has been found that sugars can develop in tubers weeks or evenmonths after environmental stress occurs.

Prior art attempts have been made to provide a means for detecting“sugar ends” in an optical sorting device. An example of one of thesedevices is seen in U.S. Patent Publication No. U.S. 2014/0056482A1 toBurgstaller et al. and which discloses a sensor unit in a machine fordetecting “sugar end” defects of potatoes, and which includes amethodology which has the steps of irradiating potatoes with at leastone light source and collecting a reflected light; and then applying atleast one classification feature to the light measurement signals takenfrom the reflected light. In the methodology as described in thatpublished application, the at least one classification featurecorresponds to a predefined “sugar end” criterion. Once the respectivepotato being sorted is classified as having a “sugar end” defect it is,thereafter, removed from further processing. It should be understoodthat the means employed in this published application for thedetermination or detection of a classification feature comprises, atleast in one form of the invention, calculating a deference curve forindividual locus points by calculating the differences between thespectral light measurement signals of the respective locus points, andthe spectral values of a referenced spectrum for a number ofwavelengths; or by calculating the differences between the nthderivative of the spectral light measurement signals of the respectivelocus points, and the nth derivative of the reference spectrum for anumber of wavelengths.

While this methodology, as discussed in the above-referenced publishedpatent application has achieved some degree of success, theimplementation of the methodology has proved, in some instances, to bedifficult or cumbersome. Consequently, the amount of potato productswhich can be processed utilizing this same technology appears to besomewhat limited in view of the complexity of the methodology as morespecifically outlined in that reference. The teachings of U.S. PatentPublication No. U.S. 2014/0056482A1 is incorporated by reference herein.

The present invention, as described hereinafter, avoids the detrimentsassociated with the prior art practices, and provides a new method ofsorting which allows food processors an improved means for detecting,and then removing agricultural products having defects in a manner notpossible, heretofore.

SUMMARY OF THE INVENTION

A first aspect of the present invention relates to a method of sortingwhich includes acquiring a multiplicity of synchronized image signalsfrom a plurality of image generating devices; generating a multiplicityof fused sensor signals by combining the multiplicity of synchronizedimage signals of the image generating devices; forming an image modelcomprising image signals previously acquired from objects of interestand defects; applying the image model to the multiplicity of fusedsensor signals, and forming a resulting object presence and defectsignals; identifying individual objects of interest with the respectiveobject presence and defect image signals; determining a spatialorientation and location of the objects of interest in each of theobject presence and defect image signals; detecting defects within theobject presence and defect image signals by comparing defect aspectsrelative to object aspects, to object images formed of the objectpresence and defect image signal; and removing the unacceptableagricultural products having defects from the product stream.

Another aspect of the present invention relates to a method of sortingwhich includes acquiring a multiplicity of synchronized image signals,each having discreet signal features, from a plurality of imagegenerating devices, and wherein the synchronized image signals representindividual agricultural products traveling in a product stream, andwhich have characteristics, and aspects, which are deemed acceptable forfurther processing, and characteristics, and aspects, which are deemedunacceptable, for further processing; generating a multiplicity of fusedsensor signals by combining the multiplicity of synchronized imagesignals by a selective synchronization of the image generating devices,and by utilizing a known position, orientation, and an operationalresponse of the respective image generating devices so as to allow thegeneration of an accurate spatial resolution of each of the agriculturalproducts travelling in the product stream, and to further align thesignal features of each of the image signals; predicting the presence ofthe acceptable, and unacceptable agricultural products in the fusedsensor signals by applying an image model previously formed from amultiplicity of image signals acquired from acceptable and unacceptableagricultural products, to the multiplicity of fused sensor signals, soas to facilitate the formation of a resulting acceptable agriculturalproduct image signal; and an unacceptable agricultural product imagesignal; identifying individual agricultural products travelling in theproduct stream as being an acceptable, or an unacceptable agriculturalproduct, by identifying one or more of a group of pixels in each of theacceptable and unacceptable agricultural product image signals;determining a spatial orientation of the identified individualagricultural products travelling in the product stream by applying aprior source of knowledge of acceptable and unacceptable agriculturalproduct aspects, characteristics and agricultural object images, to theacceptable and unacceptable agricultural product image signals;detecting unacceptable agricultural products by applying a prior sourceof knowledge of unacceptable agricultural product aspects and objectimages to the acceptable and unacceptable agricultural product imagesignals; identifying the location of the unacceptable agriculturalproducts in the acceptable and unacceptable product image signals; andremoving the unacceptable agricultural products from the product stream.

These and other aspects of the present invention will be discussed ingreater detail hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the invention are described below withreference to the following accompanying drawings.

FIG. 1 is a greatly simplified, schematic view of an apparatus which maybe utilized to practice the present methodology.

FIG. 2 is a greatly simplified, schematic view of the methodology of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

This disclosure of the invention is submitted in furtherance of theconstitutional purposes of the U.S. Patent Laws “to promote the progressof science and useful arts” (Article 1, Section 8).

The present methodology of the invention is generally indicated by thenumeral 10, and is best understood by a study of FIGS. 1 and 2,respectively. As best understood by a study of FIG. 1, the methodemploys, in one form of the invention, a conveying device which isfragmentarily shown in FIG. 1 as a conveying device 11. The conveyingdevice, as illustrated in that drawing, shows a distal, discharge end 12of a conveyor belt 11, and which further has an upper conveying surface13 which supports a product stream 14 to be inspected. The productstream 14 is formed of individual objects of interest 15, which arereleased from the distal, discharge end 12, and allowed to move, underthe influence of gravity, in a downwardly directed path of travel whichis generally indicated by the numeral 16. A source of the objects ofinterest or products which is to be inspected, and sorted, is labeled bythe numeral 17.

The product stream 14 has objects of interest or products 15 which, inone form of the invention, may include various agricultural productswhich have both acceptable features for further processing 20, orunacceptable features for processing which are generally indicated bythe numeral 21. For example, in the processing of potatoes, unacceptablefeatures 21 of a potato product would be the presence of “sugar ends” orregions of rot, which will be detected by the methodology as described,hereinafter. Positioned downstream of the distal discharge end 12, ofthe conveying device 11, are a multiplicity of image capturing devices22 which are generally shown, and which further are positioned laterallyoutwardly relative to the downwardly directed path of travel 16. Themultiplicity of imaging capturing devices 22 (which may include, forexample hyperspectral or multispectral cameras of assorted designs) areutilized, in a first step of the present method, and which includesacquiring a multiplicity of synchronized image signals 23 each havingdiscreet signal features, from a plurality of image generating devices22. The multiplicity of image capturing devices 22 produce amultiplicity of synchronized image signals 23 which are then selectivelysupplied to a first controller, and which is further generally indicatedby the numeral 24. The synchronized image signals 23 representindividual objects of interest 15 such as agricultural products whichare traveling in the product stream 14, and which have characteristicsand aspects which are deemed acceptable 20, for further processing, andcharacteristics and aspects which are deemed a defect, or unacceptable21 for further processing. The downwardly directed path of travel 16 ofthe product stream 14 passes through a downstream inspection stationwhich is generally indicated by the numeral 25, and a downstream defectremoval station 26, and which is further located elevationally, belowthe inspection station 25. The multiplicity of image capturing devices22 are positioned so as to acquire image signals 23 from the objects ofinterest 15 while they pass through the inspection station 25 in amanner well understood in the art. Assorted optical reflectors 27, andoptical combiners 28, are provided, and which co-align multiple imagecapturing devices 22. Further, well known background elements areprovided, and which additionally are positioned laterally, outwardlyrelative to the product stream 14, and which is passing through theinspection station 25.

As best seen by reference to FIG. 1, the methodology of the presentinvention includes a step of generating a source of synchronizedelectromagnetic radiation 31 which is directed towards, and reflected atleast in part from, the product stream 14, and which further is formedof the individual objects of interest 15 passing through the inspectionstation 25. As seen in FIG. 1, an electromagnetic radiation emitter 30is generally illustrated, and which is positioned laterally, outwardlyrelative to the inspection station 25. When selectively energized, theelectromagnetic radiation emitter 30 emits a source of electromagneticradiation 31 which is directed towards, and reflected at least in partby the individual objects of interest 15 passing through the inspectionstation 25. This emitter of electromagnetic radiation may producevarious wavelengths of electromagnetic radiation which enhances theability of the respective image capturing devices 22 to captureindividually unique images of the objects of interest 15 which allow forthe identification of assorted characteristics and aspects of eachobject of interest, and in particular, various defects in the productstream 14. For example, the defects could include “sugar ends” orregions of rot which are present in a potato products undergoing theinspection and sorting. As also seen in FIG. 1, an electromagneticradiation emitter control signal 32 is provided to each electromagneticradiation emitter 30. This control signal 32 causes the selectiveenergizing of each of the emitters 30. The selective energizing of theindividual electromagnetic radiation emitters 30 to achieve theaforementioned benefits of this invention, is explained in significantdetail in U.S. Pat. No. 9,266,148, the teachings of which areincorporated by reference herein. The method 10 of the present inventionincludes still another step of generating a multiplicity of fused imageand sensor signals 34 by combining the multiplicity of synchronizedimage and sensor signals 23 by a selective synchronization of the imageand sensor generating devices 22 and the respective electromagneticradiation emitters 30, and by utilizing a known position, orientationand an operational response of the respective image and sensorgenerating devices 22, so as to allow the generation of an accuratespatial resolution of each of the objects of interest 15 which aretraveling in the product stream 14, and to further align the signalfeatures of each of the image and sensor signals. As seen in FIG. 1, thecontrol signal 32 is operably controlled by the selectivesynchronization step 34. The selective synchronization 34 of the imageand sensor signals and the energizing of the electromagnetic radiationemitters 30, is performed in a manner so as to inhibit the destructiveinterference that may occur when the electromagnetic radiation emitters30, and sensors 22 are simultaneously rendered operable. In themethodology 10 of the present invention, the synchronized image andsensor signals 23 are formed by a methodology which includes the step ofselecting image signals 23 from multiple, different sensors 22. Stillfurther in the methodology 10 of the present invention the aligning ofthe signal features of each of the synchronized image and sensor signalsso as to form, at least in part, the multiplicity of fused sensorsignals 35 comprises the step of conducting a spatial registration ofthe sensor signals 23 with each other, and with an ejector controller100, and which will be discussed in greater detail, below. The highaspect spatially fused sensor signals 35 are then supplied to the firstcontroller 24 as seen in FIG. 2.

The high aspect spatially fused sensor and image signals 35 are providedto the controller 24, and to individual modules within the controller24, (FIG. 2) and which perform many of the methodology steps of thepresent invention. In this regard, the first controller 24 has a module40 which contains prior knowledge of object defects which have beenacquired from the previous inspection of similar objects of interest.This prior knowledge of the object defects 40 is provided to anothermodule 41, and which conducts supervised training of the firstcontroller 24, so as to allow the present methodology to teach itselfhow to improve the sorting reliability of the present invention 10. Thesupervised training module 41 is supplied with a portion of the highaspect, spatially fused sensor and image signals 35, and which has beenacquired from the multiple image capturing devices 22. Still further thefirst controller 24 includes a prediction module 42 which predicts thepresence of the objects of interest 15, and possible defects in thefused sensor signals 35 by applying an image model 43, which waspreviously formed from a multiplicity of image and sensor signals whichwere acquired from the objects of interest 15, and the defects, to themultiplicity of fused sensor signals 35, so as to facilitate theformation of a resulting object presence image signal 44, and defectimage signal 45, respectively. The present methodology 10 includes asecond controller 50.

The second controller 50 is operably coupled with the first controller24. Still further the second controller has a module 60 which implementsa step in the methodology 10 which includes identifying the individualobjects of interest 15 with the object presence signals 44, anddefective image signals 45, by identifying one, or more of a group ofpixels in each of the object presence and defect image signals. Stillfurther, the second controller 50, and more specifically the module foridentifying objects of interest 60, is operable to supply a signal 61 tothe module for supervised training 41, so as to allow the module forsupervised training 41 to continue to learn as the inspection processproceeds so as to increase the accuracy and sorting efficiency of thepresently disclosed methodology 10. The object presence and defectsignals 44 and 45 are supplied to other modules in the second controller50. More specifically, the second controller 50 has a module forimplementing a step which includes storing and supplying a source ofknowledge of the object aspects for use in the sorting process. Thismodule 70 supplies the stored information to another module 71, andwhich implements a step in the methodology 10 of determining a spatialorientation of the objects of interest 15 traveling in the productstream 14 by applying the prior source of knowledge of the objectaspects 70, to a multiplicity of object images which are formed of theobject presence and defect image signals 44 and 45, respectively. Stillfurther the second controller 50 includes a module which provides aprior source of knowledge of defect aspects 80, relative to objectaspects. In this regard this prior knowledge 80 is provided to a module81 for detecting defects within the unacceptable objects of interest 15by applying the prior source of knowledge 80 to object aspects 70, andto the object images formed of the object presence and defect signals 44and 45, respectively. The module for detecting defects 81, and theobject presence, and defect signals 44 and 45, generates a defect signalwhich is generally indicated by the numeral 82, and which further isitself supplied to an object removal control 90. The object removalcontrol 90 generates a signal 91 which is provided to an ejectorcontroller, which further is generally indicated by the numeral 100.Therefore, the methodology of the present invention 10 after identifyingthe location of the unacceptable objects of interest having defects 81,in the object image signals, the methodology 10 includes a step ofremoving the unacceptable objects of interest having defects from theproduct stream by means of the ejector controller 100. The ejectorcontroller operably controls an air manifold 101, (FIG. 1) ofconventional design, and which is further configured to release apressurized blast of ambient air 102, which is operable to removedefective objects of interest 21 from the downwardly descending productstream 16, so as to form a product stream having only acceptable objectsof interest 15.

Operation

The operation of the described embodiment of the present invention isbelieved to be readily apparent, and is briefly summarized at thispoint.

In its broadest aspect the present invention relates to a method ofsorting 10 which comprises a first step of acquiring a multiplicity ofsynchronized image signals 23 of individual objects of interests 15, anddefects 21, from a plurality of image generating devices 22. The methodincludes another step of generating a multiplicity of fused sensorsignals 34 by combining the multiplicity of synchronized image signals23 of the image generating devices 22. Still further the method ofsorting 10 of the present invention includes yet another step of formingan image model 43 comprising image signals 23 which were previouslyacquired from the objects of interest 15, and the defects 20. The methodincludes yet another step of applying the image model 43, to themultiplicity of fused sensor signals 34, and forming a resulting objectpresence 44, and defect signals 45, respectively. The method of thepresent invention 10 includes another step of identifying individualobjects of interest 60, with the respective object presence 44, anddefect image signals 45. The method of the present invention includesyet another step of determining a spatial orientation and location ofthe objects of interest 71 in each of the image signals 23. The methodincludes yet another step of detecting defects 81 within the objectpresence and defect signals 44 and 45, respectively, by comparing defectaspects 80 relative to object aspects 70, to object images formed of theobject presence and defect image signals 44 and 45, respectively.Finally, the present invention in its broadest aspect includes a laststep of removing 100 the objects of interest 15 having defects 21 fromthe product stream 14.

The method 10 of the present invention includes another step, andwherein the synchronized image signals 23 represent individual objectsof interest 15 such as agricultural products traveling in a productstream 14, and which have characteristics, and aspects, which are deemedacceptable for further processing 20, and characteristics and aspectswhich are deemed unacceptable for further processing 21. This is bestseen in FIG. 1. The method includes another step, and wherein thesynchronized image signals 23 are formed by a selective synchronization23 of the image generating devices 22, and by utilizing a knownposition, orientation and an operational response of the respectiveimage generating devices 22 so as to allow the generation of anaccurate, spatial resolution of each of the objects of interest 15traveling in the product stream 14, and to further align the signalfeatures of each of the image signals 23. In the present invention themethod includes a further step, and wherein the image models 43 areformed from a methodology which includes previously acquiring 40 amultiplicity of image signals 23 from known acceptable 20, andunacceptable 21 objects of interest 15, such as agricultural products,and the like. In the present methodology 10 the step of identifyingindividual objects of interest 60 further includes the step ofidentifying one or more pixel groups in each of the object presence 44,and defect image signals 45 which identify the acceptable objects ofinterest 20 or defects 21.

In the present invention the methodology 10 includes another step ofdetermining the spatial orientation 71, and location of the respectiveobjects of interest 15 and further comprises another step of developinga prior source of knowledge of object aspects 70, and which is appliedto object images which are formed of the object presence and defectsignals 44 and 45, respectively. In the present methodology 10, the stepof detecting unacceptable agricultural products, or defects in theobjects of interest 81, further comprises another step of developing aprior source of knowledge of defect aspects 80, relative to objectaspects 70, to a multiplicity of object images formed of the objectpresence and defect image signals 44 and 45, respectively. In thepresent methodology 10, and before the step of acquiring themultiplicity of synchronized image signals 23, the method furthercomprises still another step of providing a product stream 14 ofindividual objects of interest 15, such as agricultural products havingboth acceptable agricultural products 20, and unacceptable products 21,and which must be removed from the product stream 14. The methodincludes another step of passing the product stream 14 having both theacceptable agricultural products 20 and unacceptable agriculturalproducts or objects of interest 21 through an inspection station 25(FIG. 1). In the present invention, the methodology 10 includes anotherstep of acquiring the multiplicity of synchronized image signals 23, andafter the step of providing the product stream 14, generating a sourceof synchronized electromagnetic radiation 31 which is directed towardsand reflected, at least in part, from the product stream 14 of theagricultural products or objects of interest passing through theinspection station 25. In the present invention the method 10 furthercomprises a step of providing a first controller 24 which predicts thepresence of the objects of interest 15 and defects 21 in the fusedsensor signals 35, and which further applies the image model 43 to atleast some of the multiplicity of fused sensor signals 35. The methodfurther comprises another step of providing a second controller 50, andwhich identifies individual objects of interest 15, and defects 21, inthe object presence and defect image signals 44 and 45; and furtherdetermines the spatial orientation 71 of the identified objects ofinterest 15 and defects 21 traveling in the product stream 14. Themethod 10 also includes another step of identifying the defects 81, andthe defect image signal 82 identifies the location of the defects in thedefect image signals. An unacceptable agricultural product image signalis provided to the third controller or object removal control 90. In thepresent methodology 10, the third controller 90 is controlled, andcoupled with, and renders operational an ejector 101. The thirdcontroller 90 is coupled in signal receiving relation relative to theunacceptable agricultural product image or defect signal 82 which isgenerated by the second controller 50, and further renders the ejector101 operational by means of the ejector controller 100 to remove theunacceptable agricultural product or any defects 21 from the productstream 14, and which is passing by, or through, the defect removalstation 26.

More specifically the methodology 10 of the present invention furtherincludes a step of acquiring a multiplicity of synchronized image andsensor signals 23, each having discreet signal features, from aplurality of image generating and sensor devices 22. The synchronizedimage and sensor signals 23 represent individual objects of interest 15,such as agricultural products, which are traveling in a product stream14, and which have characteristics and aspects which are deemedacceptable for further processing 20, and characteristics and aspectswhich are deemed a defect and unacceptable 21 for further processing.The present method includes another step of generating a multiplicity offused image and sensor signals 34 by combining the multiplicity ofsynchronized image and sensor signals 23 by a selective synchronizationof the image and sensor generating devices 22, and by utilizing a knownposition, orientation, and an operational response of the respectiveimage and sensor generating devices 22 so as to allow the generation ofan accurate spatial resolution 35 of each of the objects of interest 15traveling in the product stream 14, and to further align the signalfeatures of each of the image and sensor signals 23. The method includesstill another step of predicting the presence 42 of the objects ofinterest 15, and possible defects 21, in the fused image and sensorsignals 35 by applying an image model 43 which is previously formed froma multiplicity of image signals 35, and which are further acquired fromthe objects of interest 15 and defects 21, to the multiplicity of fusedsensor signals 35 so as to facilitate the formation of a resultingobject presence image signal 44, and a defect image signal 45. Themethod 10 includes another step of identifying the individual objects ofinterest 15, with the object presence and defect image signals 44 and45, by identifying one or more of a group of pixels in each of theobject presence and defect image signals 44 and 45, respectively. Themethod 10 of the present invention further includes another step ofdetermining a spatial orientation 71 of the objects of interest 15traveling in the product stream 14 by applying a prior source ofknowledge 70 of the object aspects to a multiplicity of the objectimages which are formed of the object presence and defect image signals44 and 45, respectively. The method includes yet another step ofdetecting defects 81 within the unacceptable objects of interest 15 byapplying a prior source of knowledge 80 of defect aspects relative toobject aspects, to the object images formed of the object presence andobject defect signals 44 and 45, respectively. The method includes stillanother step of identifying the location 81 of the unacceptable objectsof interest having defects in the object image signals; and yet anotherstep 100 of removing the unacceptable objects of interest 15 havingdefects from the product stream 14 so as to provide a resulting uniformproduct stream.

In the methodology of the present invention 10 the discreet signalfeatures of the multiplicity of synchronized image signals 23 areselected from the group comprising signals generated by any one or moreof individual hyperspectral or multispectral imagers or scanners 22which are employed in the apparatus and which are schematicallyrepresented in FIG. 1.

Still further the synchronized image signals 23 are formed by amethodology which includes a step of conducting a spatial registrationof the respective image signals. In addition to the foregoing, theaspects and characteristics of the objects of interest 15, and which aredeemed acceptable for further processing are selected from individualproducts 15 having a known, and acceptable qualities. Still further, thecharacteristics of the objects of interest 15 which are deemedunacceptable for further processing are selected from the groupcomprising individual products 15 having known unacceptable qualities.Moreover, the aligning of the signal features of each of thesynchronized image signals 23 so as to form, at least in part, themultiplicity of fused image or sensor signals 35 comprises another stepof conducting a spatial registration of the respective sensors 22 witheach other, and with the ejector controller 100.

The methodology 10 of the present invention further includes yet othersteps which are directed to the formation of the image model 43. In thisregard, the image model 43 is formed by a methodology which includes astep of utilizing a standard classification algorithm such as a partialleast square algorithm (PLS). In addition to the foregoing, the priorsource of knowledge of the object aspects 70, which is supplied to themultiplicity of object images, and which are further used to determinethe spatial orientation of the identified objects of interest 71 isformed by a methodology which includes the steps of conducting an objectshape analysis; and conducting an object aspect measurement. Moreover,the prior source of knowledge of the defect aspects 80, and which areapplied to the multiplicity of object images formed by the presentmethodology comprises the step of qualifying unacceptable pixel groupsfound in the image signals with object regions identified in objectaspects. Additionally, the step of removing the unacceptable objects ofinterest 90 from the product stream 14 further comprises the step ofremoving an unacceptable portion of an object of interest 15, from anacceptable portion of the same object of interest 15.

As should be understood, and in the present methodology 10, and beforethe step of acquiring the multiplicity of synchronized image signals 23,the method 10 further includes a step of providing a product stream 14of individual objects of interest 15 which have characteristics andaspects of both acceptable 20, and unacceptable objects of interest 21,and passing the product stream 14 having both the acceptable andunacceptable objects of interest 20 and 21 through an inspection station25. In addition to the foregoing the methodology further includes,before the step of acquiring the multiplicity of synchronized imagesignals 23, and after the step of providing the product stream 14,generating a source of synchronized electromagnetic radiation 31 whichis directed towards, and reflected at least in part from, the productstream 14 which is formed of the objects of interest 15 passing throughthe inspection station 25. In addition to the foregoing, the method ofthe present invention 10 includes yet another step of providing a firstcontroller 24 which predicts the presence of the objects of interest 15,and defects 21 in the fused sensor signals 35, and which further appliesthe image model 43 to the multiplicity of fused sensor and image signals35. The method 10 further includes still other steps of providing asecond controller 50 which identifies individual objects of interest 15,and defects 21, in the product stream; determines the spatialorientation of the identified individual objects of interest 71traveling in the product stream 14; detects the objects of interest 15;identifies the location of the defects 21 in the object presence anddefect image signals 44 and 45, and further generates a signal 82 whichindicates the presence and location of the defect(s) 21 in the productstream 14. The method 10 of the present invention includes yet anotherstep of providing a defect removal station 26, and positioning thedefect removal station downstream of the inspection station 25 (FIG. 1).The method includes another step of providing an ejector 101, andpositioning the ejector 101 in the defect removal station 26, and whichis effective, when made operational, to remove the unacceptable objectsof interest 21 from the product stream 14. The method 10 includes yetanother step of providing an ejector controller 100 which iscontrollably coupled with, and renders operational the ejector 101. Athird controller 90 is coupled in defect signal 82 receiving relationrelative to the second controller 50, and which further renders theejector controller 100 operational to cause the ejector 101 to removethe unacceptable objects of interest 21 from the product stream 14 (FIG.1). Therefore it will be seen that the present invention provides aconvenient means for detecting unacceptable objects of interest in aproduct stream, and which may include agricultural products such aspotatoes having agricultural defects such as “sugar ends,” in aparticularly efficient fashion which has not been available in automatedsorting devices utilized, heretofore. The present methodology isefficient, operates reliably, and further has an inventive feature whichallows it to self-learn so as to increase the efficiency and reliabilityof the sorting operations of a device employing same.

In compliance with the statute, the invention has been described inlanguage more or less specific as to structural, and methodicalfeatures. It is to be understood, however, that the invention is notlimited to the specific features shown and described since the meansherein disclosed comprise preferred forms of putting the invention intoeffect. The invention is, therefore, claimed in any of its forms ormodifications within the proper scope of the appended claimsappropriately interpreted in accordance with the Doctrine ofEquivalence.

We claim:
 1. A method of sorting comprising: providing a product streamof individual objects of interest which are non-uniform agriculturalproducts, and passing the product stream through an inspection station;acquiring a multiplicity of synchronized image signals of the individualobjects of interest from hyperspectral or multispectral image generatingdevices, each of the multiplicity of synchronized image signals havingdiscreet signal features, and wherein the acquired multiplicity ofsynchronized image signals represent the individual agriculturalproducts traveling in the product stream, and the individualagricultural product's characteristics, and aspects, which are deemedacceptable for further processing, and the individual agriculturalproduct's characteristics, and aspects, which include features that areinvisible to visible spectrum optical detection, and which are deemedunacceptable, for further processing; providing an image model formedfrom a multiplicity of previously acquired synchronized image signalsfrom individual agricultural products having acceptable characteristicsand aspects, and from individual agricultural products havingunacceptable characteristics and aspects that include features that areinvisible to visible spectrum optical detection; generating amultiplicity of fused image and sensor signals by combining themultiplicity of synchronized image signals by a selectivesynchronization of the image generating devices, and by utilizing aknown position, orientation, and an operational response of each of theimage generating devices to generate an accurate spatial resolution ofeach of the individual agricultural products travelling in the productstream, and to further align the discrete signal features of each of themultiplicity of synchronized image signals; providing a controller thatimplements control function; providing a first control function topredict the presence of acceptable, and unacceptable characteristics inthe individual agricultural products in the product stream by applyingthe image model to the multiplicity of fused image and sensor signals,to facilitate formation of an object presence image signal, and tofacilitate the formation of a defect image signal; providing a secondcontrol function to identify individual objects of interest travellingin the product stream as being an acceptable agricultural product, or asbeing an unacceptable agricultural product, by identifying one or moreof a group of pixels in each of the object presence image signals anddefect image signals which identify objects of interest or defects;determining a spatial orientation and location of the identifiedindividual objects of interest travelling in the product stream byapplying a prior source of knowledge of acceptable agricultural productaspects and unacceptable agricultural product aspects, to the objectpresence image signals and to the defect image signals; detectingdefects within the unacceptable individual objects of interest in theproduct stream by applying a prior source of knowledge of unacceptableagricultural product aspects relative to acceptable object aspects toobject images formed of the object presence image signals and the defectimage signals; updating the image model with additional and subsequentlyacquired object presence image signals and defect image signals;identifying the position and location of the unacceptable individualobjects of interest in the product stream; and providing a third controlfunction which renders operational an ejector, and wherein the thirdcontrol function is coupled in signal receiving relation relative to anunacceptable agricultural product defect signal which is generated bythe second control function, and which further renders the ejectoroperational to remove unacceptable agricultural products from theproduct stream.
 2. A method as claimed in claim 1, and wherein the stepof determining the spatial orientation and location of the respectiveobjects of interest further comprises developing a prior source ofknowledge of object aspects which is applied to object images, and whichare formed of the object presence and defect signals.
 3. A method asclaimed in claim 1, and wherein the step of detecting unacceptableagricultural products or defects further comprises developing a priorsource of knowledge of defect aspects, relative to object aspects in amultiplicity of object images formed of the object presence, and defectimage signals.
 4. A method as claimed in claim 1, and wherein the secondcontrol function identifies individual objects of interest and defectsin the object presence and defect image signals; determines the spatialorientation of the identified objects of interest and defects travellingin the product stream; identifies the defect in the defect image signal;identifies the location of the defect in the defect image signal; andgenerates an unacceptable agricultural product image signal.
 5. A methodas claimed in claim 1 and wherein, the generation of an accurate spatialresolution of each of the individual agricultural products in theproduct stream further includes the step of aligning the discrete signalfeatures of each of the multiplicity of synchronized image signals alonga major axis of each of the individual agricultural products so that endportions of each of the individual agricultural products areidentifiable; and wherein the step of identifying one or more of a groupof pixels in each of the individual image signals further includes thestep of identifying a location of the one or more group of pixels asoccurring at one of the end portions of the individual agriculturalproducts, or not at one of the end portions of the individualagricultural products; and when the identified one or more of the groupof pixels is identified as occurring at a designated position along themajor axis of the individual agricultural product, the product isidentified as unacceptable and is removed from the product stream by theejector, and when the one or more of a group of pixels is identified asnot occurring at a designated position along the major axis of theindividual agricultural product, the individual agricultural product isidentified as acceptable.
 6. A method of sorting as claimed in claim 1and wherein the unacceptable characteristics and aspects arephysiological tuber disorders.
 7. A method of sorting as claimed inclaim 1 and wherein the unacceptable characteristics and aspects areconcentrations of sugars.
 8. A method of sorting as claimed in claim 1and wherein the unacceptable characteristics and aspects areconcentrations of reducing sugars.
 9. A method of sorting as claimed inclaim 1 and wherein the unacceptable characteristics and aspects aresugar ends.
 10. A method of sorting as claimed in claim 5 and whereinthe unacceptable characteristics and aspects are sugar end defects; andthe sugar end defects occur at an end portion along the major axis ofthe object of interest.
 11. A method of sorting, comprising: acquiring amultiplicity of synchronized image and sensor signals, each havingdiscreet signal features, from the image generating devices and sensors,and wherein the synchronized image and sensor signals representindividual objects of interest such as agricultural products which aretraveling in a product stream, and which have characteristics, andaspects which are deemed acceptable for further processing, andcharacteristics and aspects which are deemed a defect and which includefeatures that are invisible to visible spectrum optical detection, andunacceptable, for further processing; generating a multiplicity of fusedsensor signals by combining the multiplicity of synchronized image andsensor signals by a selective synchronization of the image generatingdevices and sensors, and by utilizing a known position, orientation, andan operational response of the respective image generating devices andsensors so as to allow the generation of an accurate spatial resolutionof each of the objects of interest products travelling in the productstream, and to further align the signal features of each of the imageand sensor signals; predicting the presence of the objects of interest,and possible defects in the fused sensor signals by applying an imagemodel previously formed from a multiplicity of image signals which wereacquired from the objects of interest, and the defects, to themultiplicity of fused sensor signals so as to facilitate the formationof a resulting object presence image signal; and a defect image signal;identifying the individual objects of interest with the object presence,and defect image signals, by identifying one or more of a group ofpixels in each of the object presence, and defect image signals;determining a spatial orientation of the objects of interest travellingin the product stream by applying a prior source of knowledge of theobject aspects to a multiplicity of object images which are formed ofthe object presence, and defect image signals; detecting defects withinunacceptable objects of interest by applying a prior source of knowledgeof defect aspects relative to object aspects, to the object imagesformed of the object presence, and defect signals; identifying thelocation of the unacceptable objects of interest having defects in theobject image signals; and removing the unacceptable objects of interesthaving defects from the product stream.
 12. A method as claimed in claim11, and wherein the discreet signal features of the multiplicity ofsynchronized image signals are selected from the group of image signalsprovided by a hyperspectral or multispectral imager and/or laserscanner.
 13. A method as claimed in claim 11, and wherein thesynchronized image signals are formed by a methodology which includes astep of spatially registering the respective image signals.
 14. A methodas claimed in claim 11, and wherein the aspects and characteristics ofthe objects of interest which are deemed acceptable for furtherprocessing are selected from individual products having known acceptablequalities.
 15. A method as claimed in claim 11, and wherein the aspectsand characteristics of the objects of interest which are deemedunacceptable for further processing are selected from individualproducts having known unacceptable qualities.
 16. A method as claimed inclaim 11, and wherein the aligning of the signal features of each of thesynchronized image signals so as to form, at least in part, themultiplicity of fused sensor signals comprises a partial registration ofthe image signals with each other, and with an ejector control function.17. A method as claimed in claim 11, and wherein the image model isformed by a methodology which includes a step of utilizing a standardclassification algorithm.
 18. A method as claimed in claim 11, andwherein the prior source of knowledge of the object aspects which isapplied to the multiplicity of object images, and which is used todetermine the spatial orientation of the identified objects of interestis formed by a methodology which comprises a step of conducting anobject shape analysis; and conducting an object aspect measurement. 19.A method as claimed in claim 11, and wherein the prior source ofknowledge of the defect aspects, and which is applied to themultiplicity of object images, is formed by the methodology whichcomprises a step of qualifying unacceptable pixel groups found in theobject images, with object regions identified in the object aspects. 20.A method as claimed in claim 11, and wherein the step of removing theunacceptable objects of interest from the product stream furthercomprises a step of removing an unacceptable portion of an object ofinterest from a remaining acceptable portio of the same object ofinterest.
 21. A method as claimed in claim 11, and wherein the firstcontrol function predicts the presence of the objects of interest, anddefects in the fused sensor signals, and which further applies the imagemodel to the multiplicity of fused sensor signals.
 22. A method asclaimed in claim 21, and wherein the second control function whichidentifies individual objects of interest and defects travelling in theproduct stream determines the spatial orientation of the identifiedindividual objects of interest travelling in the product stream; detectsthe objects of interest; identifies the location of the defects in theobject presence and defect image signals; and generates a signalindicating the presence and location of the defect in the productionstream.
 23. A method as claimed in claim 22, and further comprising:providing a defect removal station, and positioning the defect removalstation downstream of the inspection station; providing an ejector, andpositioning the ejector in the defect removal station, and which iseffective, when made operational, to remove the identified unacceptableobjects of interest from the product stream; and the third controlfunction which is controllably coupled with, and renders operational theejector, and wherein the third control function is coupled in defectsignal receiving relation relative to the second control function, andwhich further renders the ejector operational to remove the unacceptableobjects of interest from the product stream.
 24. A method of sorting asclaimed in claim 11, and wherein the unacceptable characteristics andaspects are physiological tuber disorders.
 25. A method of sorting asclaimed in claim 11, and wherein the unacceptable characteristics andaspects are concentrations of sugars.
 26. A method of sorting as claimedin claim 11, and wherein the unacceptable characteristics and aspectsare sugar ends.